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Abstract 


The United States is in a period of reckoning from which institutions of higher education are not exempt. Rather, we exist 
at the intersection of the chaos wrought by the novel coronavirus and the spread of outrage about systemic racism beyond 
Black, Indigenous, and People of Color (BIPoC) communities. The resulting almost universal sense of loss and uncertainty 
leads many of us to believe that individuals have little-to-no power to inform, never mind implement, change. Systemic 
problems require systemic solutions. There can be no return to “normal.” We instead invite our assessment colleagues 
to question the unexamined assumptions which underlie our heretofore taken-for-granted approaches to assessing and 
documenting our students’ learning; to reconnect with their foundational beliefs and values; and to fully engage with the 
uncertainty and complexity of the current moment. This paper offers readers a developmental approach for reflection, 
identifying potential leverage points, and intentionally creating a new assessment future which proactively includes a// of 
out students. 
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There Is No Return to Normal: Harnessing Chaos 
to Create Our New Assessment Future 


We are living in historic times. The global health pandemic sparked by the novel 

coronavitus (hereafter: COVID-19) continues to wreak havoc in every corner of the 

world. When COVID-19 hit the United States early in 2020, institutions of higher 

education, not known to be early adopters as a general rule, were forced to respond 

immediately to a crisis of previously unimagined magnitude. Institutions adopted a [SS 

triage model, which left decision makers with little time for reflection. With little-to-no The stark illumination of the 

warning, final exams and courses were pushed to emergency remote teaching contexts. deep structural fissures and 

Staff and faculty packed their laptops, books, and other resources to resume their work systemic inequities of our 

in home offices and at dining room tables. Students were asked to move out of campus society became too obvious 

residences, many with only a few days’ notice. The disruptions of daily life from going to _ to ignore. We have entered 

work and school, gathering for rituals such as birthdays and commencements, and seeing —_a period of reckoning and 

friends and family, contributed to a shared sense of disequilibrium. higher education institutions 
are not immune. 

Compounding the anxiety caused by the initial dislocation of higher education, COVID-19 

infection and death rates numbers in the US. continued to climb in March and eventually 

hit one million cases by April 28, 2020 Johns Hopkins University & Medicine, 2020). 

Much of the country was practicing physical distancing, sheltering in place, and wearing 

masks. Schools and colleges had already shifted to emergency remote teaching and 

learning, with only essential businesses permitted to remain open. Coronavirus required 

students, faculty, and staff to navigate overlapping stressful transitions simultaneously. 


The disproportionate numbers of Black, Indigenous, and People of Color (BIPoC) dying 
from COVID-19 and the communities impacted the hardest revealed the sociopolitical 
and economic fissures of contemporary America. At the same time, the highly visible 
and reported murders of Ahmaud Arbery, Breonna Taylor, and George Floyd, swept the 
nation and became a tipping point for communities of color that have been protesting 
injustices for decades. The stark illumination of the deep structural fissures and systemic 
inequities of our society became too obvious to ignore. We have entered a period of 
reckoning and higher education institutions are not immune. 


The intersection of the chaos wrought by the pandemic and the emerging racial 
awakening, for some, of decades of existing racial inequities and police brutality across this 
country constitutes an opportunity for deeper reflection as campus leaders. Confronting 
the seemingly Sisyphean task of examining institutional policies and practices within 
beloved academic cultures feels overwhelming at times. No single implicit bias or anti- 
discrimination workshop or webinar is sufficient to support the change needed. 


These ate systemic problems. They require systemic solutions. As the uncertainty of 
COVID-19 and racial reckoning is navigated, this paper challenges us all to take an 
anthropological approach to examining assessment culture and systems and emerge with 
new ways of being and doing that will deepen a commitment to equitable practices for 
students at your institution. 


This paper invites assessment professionals to reflect on their assessment practices with 
the goal of emerging out of this uncertain and complex time in higher education better 
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positioned to engage in the type of assessment work that is needed. This is a time for 
deep reflection, courageous deconstruction, and intentional rebuilding in alignment with 
the world in which we live and the learners whom we serve. We begin the reflective 
process by looking back to how the assessment movement started in order to locate 
where we ate today. 


Assessment Rewind: How We Got Here 


Prior to the rise of the assessment movement in the mid-1980s, decades of research in 
collegiate learning, models and taxonomies of outcomes, student growth and development 
studies of the college experience, and the rise of program evaluation as action research 
had already existed (Ewell, 2002). The concept of mastery learning that started in 
elementary and secondary education first became applicable in the mid-1960’s among 
post-secondary education’s adult and professional education programs. Establishing 
outcomes and evaluating student attainment of achievement and competencies were 
critical to these types of programs. Assessment practitioners, trained and equipped to 
evaluate authentic student evidence, became a role in higher education that demonstrated 
how the theory of assessment in higher education could be practiced and operationalized 


(Ewell, 2002, p. 6). 


As far back as assessment practitioners have existed in post-secondary education, so has 
the tension of the definition of assessment. Assessment has historically been associated 
with three traditions and purposes: 1. Mastery learning and development over time with 
continuous feedback (the word assessment dertves from the Latin and is the past participle 
of assidere/adsidere “to sit beside’’); 2. Large-scale assessment focused on the evaluation 
of institutional performance in response to accountability; and 3. Program evaluation 
with the goal of improving pedagogy and curricula (what we might call “closing the loop” 
today). 


From the 1980s through the 1990s, assessment efforts surged across multiple corners 
of higher education, so much that some referred to it as the “assessment movement” 
(Ewell, 2002). In 1992, the founding voices of what we know today as learning outcomes 
assessment identified the Principles of Good Practice for Assessing Student Learning (Astin, 
Banta, Cross, El-Khawas, Ewell, Hutchings, Marchese, Mcclenney, Mentkowski, Miller, 
Moran, & Wright, 1992). 


In just a few decades, the assessment movement identified by Ewell (2002) grew into 
a recognized, if not always welcome, discipline within higher education. Situated to 
meet external demands for accountability, the assessment Conversation’ grew beyond 
compliance into a curiosity-driven, student-centered process of inquiry and research. We 
have our assessment forbearers to thank for their efforts to reframe assessment as a 


generative, collaborative process dedicated to improving student learning. Nicholas and 
Slotnick (2018) refer to this development as the “Emergence of Institutional Assessment 
Professionals” (p. 5) and describe the resulting increase of conferences, journals, and 
associations through which assessment professionals can develop individual capacity to 
ask and pursue questions about student learning. As the assessment movement forges 
ahead, it is difficult to ignore this moment in time as a possible inflection point for higher 


‘Per Gee (2014), we use the Big-C Conversation(s) to indicate a metaphorical discussion among members of a 
discourse community or social group—in this case assessment professionals—about topics of interest to the group. 
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education and the field of assessment. If the pandemic and awakening to systemic racism 
has illuminated anything, it is that assessment also needs to be responsive in this moment. 


Assessment Professionals: Navigating Disruption 


Assessment is about inquiry and action: how we know what our students are able to 
demonstrate; what we can learn by examining direct evidence of learning. We facilitate 
conversations about what it means to know something. We encourage faculty to examine 
how their instructional practices create appropriate opportunities for students to 
demonstrate their learning. The answers to these and other questions constitute the heart 
of the stories we tell about assessment on our campuses. The global pandemic altered the 
content of the questions. Along with our colleagues in teaching and learning centers, we 
asked ourselves questions such as: 


* How can we ensure that students’ learning is not completely disrupted by the 
rapid shift of instructional contexts? 

¢ What will faculty need to know in order to embed assessment into plans for 
instructional continuity? 

¢ What tools are already in place to provide some semblance of continuity? 


To answer these new questions, we turned to our community. We sought solace in 
listservs and reached out to assessment colleagues to ask how they were responding 
to changes. We contributed to dynamically evolving lists of assessment resources. We 
registered for webinars and we surveyed! We dialogued online and wrestled with each 
other’s ideas about how to maintain learning outcomes assessment during a pandemic. 


We did not expect, but ultimately were not surprised, to discover that the questions had 
changed yet again, this time in disposition. We heard colleagues talking about maintaining 
assessment reporting schedules and asking whether they “s#// need to do assessment this year?” 
or whether regional accrediting agencies were “s#// requiring us to do program review?” Even 
as out world was falling apart, many members of our assessment community wanted to 
“Just keep moving” or “get back to normal.” 


In the report of the NILOA COVID-19 Survey, Jankowski (2020a) presents findings 
that while 97% of respondents said their institution made some kind of change in the 
spring, 75% of respondents felt the changes would not negatively impact the culture 
of assessment on their campuses. It’s no surprise then, that only 35% of respondents 
modified assessment report deadlines. While assessment professionals were adapting to 
the evolving circumstances, there was a commitment to still maintain some sense of 
normalcy and avoid any negative impact on the culture of assessment on their campus. 
But, how do we know that the culture of assessment and the normalcy we previously took 
for granted actually served our students well? Further, as we navigate the unavoidable 
changes that will occur as a result of COVID-19 and the call for anti-racism at our 
institutions, is our previously maintained culture of assessment what is needed in moving 
forward? 


According to Inside Higher Ed. (2020), 90% of university presidents expressed concern 
about the inequitable impact COVID-19 has had on minoritized students, while less than 
30% of assessment professionals responding on the NILOA survey strongly agreed that 


How do we know that the 
culture of assessment and the 
normalcy we previously took 
for granted actually served our 
students well? Is it possible 
that higher education has not 
been paying attention? Is it 
possible that higher education 
is a microcosm of what we 
have been witnessing in the 
outrage among BIPoC across 
the nation, exhausted from 
fighting systemic inequities? 
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equity, student voices, and listening to those voices occurred before making changes. Is 
it possible that higher education has not been paying attention? Is it possible that higher 
education is a microcosm of what we have been witnessing in the outrage among BIPoC 
across the nation, exhausted from fighting systemic inequities? 


In these days of crises, we are experiencing collective grief for the rituals through which 
we know ourselves. Boss (2020) refers to it as ambiguous loss. Contrary to earlier 
constructions of grief as a series of stages through which mourners move, ambiguous 
loss holds no promise of resolution. We may never fully heal our collective ambiguous 
loss. There is no normal to which we can return. But, if we ate to serve out learners well 
moving forward, the pre-COVID-19 “normal” is no longer an option. 


As we navigate this liminal space, we invite assessment professionals to seize this moment 
for reflection, questioning, and possibly disrupting their current assessment infrastructure 
and practices, with the goal of emerging better positioned to create new pathways that 
reflect our commitment to equity and inclusivity. 


A New Way Forward 


To prepare ourselves to actively create post-COVID-19 pathways, we invite assessment 
professionals to ground themselves in their first principles, as a way to lay a firm 
foundation for the work of assessment. First principles thinking is a questioning process 
that requires one to reveal underlying assumptions to the point where you are left with 
the foundational truth of a situation. It adopts a scientific approach by uncovering layers 
of inherited or adopted ways of thinking until what remains is the foundational truth 
of a situation. Aristotle defined a first principle as “the first basis from which a thing 
is known” (Met. 1013a14—15, as cited by Irwin, 1989, p. 3). First principles challenge 
beliefs and their undergirding assumptions until one reaches the underlying truth. It’s a 
way to deconstruct how we’ve built assessment practices, narratives, and discourses, in 
order to reconstruct them in alignment with our values and core principles. This is a call 
to engage in critical consciousness to create assessment practices that demonstrate our 
acknowledgment of the inequities that have always existed and our intention to no longer 
sit passively on the professional sidelines. 


When used as a developmental approach, first principles brings clarity and consistency to 
our work as we reconstruct based on what we know or are reasonably sure is true. The 
three steps involved in engaging in first principles thinking include: 


1. Identify assumptions 
2. Break down the problem to fundamental basics 
3. Build new solutions 


Step 1: Identify Assumptions 

Davidson (2017) introduced the concept of “legacy assumptions” in The New Education: 
How to Revolutionize the University to Prepare Students for a World in Flux. Legacy assumptions 
are the received practices and beliefs we inherit. They are often so pervasive and ingrained 
that they seem natural and beyond question. 
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A first step in revolutionizing higher education is being aware of these legacy 
assumptions. A second step is recognizing other models that respond to different 
assumptions. It is challenging to rethink the ways structures and methods might 
actually hinder mission. It is rare for any institution to examine its assumptions 
this deeply. Yet it is essential if we are going to revolutionize our traditional 
colleges and universities. (Davidson, 2017, p. 72). 


As assessment practitioners, when was the last time we challenged our assumptions 
about the role and purpose of assessment, or the methods and structures upon which 
assessment was built? When was the last time we scrutinized our rhetoric and practices 
around assessment and whether they unintentionally perpetuate inequality? As assessment 
professionals nested within institutional cultures that are presently confronted with 
heightened inequities illuminated during this time, we have the opportunity to consider 
how and to what degree the principles of assessment first articulated in 1992 continue 
to inform current values, beliefs, and actions, and whether new or additional principles 
need to be established. 


Dichotomized perspectives of assessment often depict our work as being either for 
accountability and compliance or improvement. How often have we reported on whether 
programs have submitted or not submitted their assessment reports, as if submission was 
the ultimate goal? Further, what underlying assumptions are we operating from when we 
claim a culture of assessment exists on our campuses and support our claims based on 
yearly percentages of report submissions? 


Whether you’ve been involved in assessment work for a year or decades, we can learn from 
examining inherited practices and habits of mind. There is no doubt that students today 
differ from students of twenty-five years ago. But, to what degree have our perspectives 
about students changed and how are we attuned to those changes? Leaderman and 
Polychronopoulos (2019) assert that: 


[O]one challenge in assessment occurs when faculty or staff get stuck in a pattern 
of maintaining the status quo through continuing to conduct assessment tasks that 
are easy to do but have consistently lacked value for the department or program, 
and have not led to usable results. This type of resistance can make it difficult to 
implement change.” (p. 35) 


In order to build an appropriate post-COVID-19 assessment culture, it is critical to 
begin by identifying assumptions about assessment that you and your team have held 
onto or have heard others express. For this deconstructive process to begin, create a 
judgment-free zone for yourself and your colleagues. Be honest with yourself. Here are 
some questions to get you started. 


¢ What do I believe about the purpose of assessment? 

¢ Why do I think that is the purpose of assessment? 

* How do I know this is true? What if I thought the opposite? 
¢ What might someone unfamiliar with assessment think? 

* How do I know I am correct? 


As assessment practitioners, 
when was the last time we 
challenged our assumptions 
about the role and purpose 
of assessment, or the 
methods and structures 
upon which assessment 

was built? When was the 

last time we scrutinized our 
rhetoric and practices around 
assessment and whether they 
unintentionally perpetuate 
inequality? 
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Allow yourself and your team to process the degree to which unexamined assumptions 
have served students and their learning; and promoted or hindered acceptance of 
assessment at your institution. Examine the relationships between assessment practices 
and various stakeholders within the institution. During this exercise, you might find that 
the stories you have told yourself and others have been repeated so often that they have 
become natural scripts in the work we do. Some of these scripts might sound like: 


* Our assessment reporting must occur regardless of circumstance (or pandemics!) 
in order to maintain a culture of assessment. 

* If we don’t have assessment reports submitted, we won’t know if learning has 
happened during COVID-19. 

* Our assessment reports demonstrate to ourselves and our accreditors that we have 
a culture of assessment. 

¢ We can no longer assess learning because we moved from in-person to unplanned 
remote instruction. 


In addition, it is more important than ever that we consider the potential and unintended 
messages we communicate when we accept and repeat fables or stories about faculty 
(“they don’t care” about assessment), students (we must prevent them from cheating now 
that we’re online), and assessment itself (“I know you hate it, but we need your report 
anyway.”’) The last story is particularly pernicious as it perpetuates a separateness between 
teaching and assessment. Are we, as Jankowski (2017) suggests, “selling something that 
faculty would not be interested in to begin with” (p. 9)? What legacy assumptions do 
we reify when we talk about increasing “buy-in” in assessment activities and how do we 
begin to disrupt these legacy assumptions that hinder the purpose of assessment moving 
forward? We suggest beginning with acknowledging and recognizing the assumptions 
that have gone unquestioned. 


Step 2: Break Down the Problem to Fundamental Basics 

The next step in the first principles development approach moves from identifying 
assumptions to examining the fundamental basics and absolute truths that will serve as the 
foundation for the assessment work you do. You may experience a sense of dissonance if 
your examination reveals that previously unexplored assumptions have in fact interfered 
with the core values and principles of your work. This is the time to codify the values that 
drive your work. While the compliance versus continuous improvement narrative has 
existed for decades, we question whether these stories are sufficient for the assessment 
movement going forward. Specifically, how does the dichotomy keep us in a useless 
debate that only serves to stagnate our work, marginalize would-be collaborators and 
colleagues, and detract us from our focus on creating equitable learning opportunities? 
We must ask ourselves, “What language do we have for understanding and describing 
students’ lived experiences now that did not exist thirty years ago? What do we know 
about today’s learners? What do we not yet know?” Failing to ask and answer these and 


other questions could have deleterious long-term consequences for assessment writ large, 
assessment practitioners, and perhaps most importantly for the students on whose behalf 
we do our work. 


The pandemic has exacerbated pre-existing inequities, leaving us to (re)consider the 
fundamental purposes of assessment at our institutions. To help get the conversation 
started, consider how one or more of the following foundational statements might (re) 


round work in assessment: : ; 
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¢ Assessment is about students and student learning. 

¢ Assessment is an opportunity to shift the paradigm of teaching to learning (Barr 
& Tagg, 1995). 

¢ Assessment creates space for critical reflection and action. 

¢ Assessment opens up an opportunity to engage in equity-minded sense-making 
(McNair, Bensimon, & Malcom-Piqueux, 2020). 

¢ Assessment data is valuable when it is used. 

* Student voice is integral to the work of assessment (Montenegro & Jankowski, 
2020). 


Exploring these assessment fundamentals and core truths provides an opportunity 
to “uncover underlying tendencies towards different philosophies of assessment” 
(Jankowski, 2020b, p. 1) and align assessment philosophies with practices. 


Step 3: Build New Solutions 

The third part of the first principles development approach moves from reflection, 
critical analysis, and deconstruction, toward new ways of being and doing. This third 
aspect rests on foundational truths and principles that emerged as a result of engaging 
in the previous process. New solutions must be aligned and undergirded by the prior 
established core truths. There is little appetite for disconnect between what institutions 
purport to stand for and the policies and practices that play out. We are witnessing a call 
for greater accountability in our institutions, as students have demanded administrators 
move beyond lip service and messages of support for Black lives to actual systemic 
change, with some demanding their institutions cut ties with local police (Whitford & 
Burke, 2020). One might ask, “What does this have to do with building and fostering a 
new culture of assessment?” 


What We Say Matters 

A 2017 NILOA survey revealed that effective communication about student learning is 
both an opportunity and a challenge for assessment professionals (Jankowski, Timmer, 
Kinzie, & Kuh, 2018). Intentional examination of one’s own language is necessary in 
building or reconstructing new assessment practices. This work requires us to go beyond 
“changing words to make our language more enticing to faculty and other staff to get 
them ‘on board’ with assessment; it is about matching our language with what we really 
want to achieve through engaging in meaningful assessment processes” (Jankowski, 
2017, p. 10) or what we have chosen as our foundational assessment truths. 


As you seek to build new solutions, examine language use, both formal and informal, 
to illuminate whether your assessment truths are helped or hindered by your discourse. 
Following are prompts and questions to get you started. 


¢ Open the last ten emails you sent about assessment. What do you notice about 
the content or topics? What was the purpose for each of the messages? What 
perspective or tone might you have conveyed unintentionally? How do you know 
if your recipients perceive the message in the way you intended? 

* How often do you contact faculty when you need something from them or want 
them to submit a report, answer your questions, check items off of lists? 

* How often do you contact campus partners to ask for their stories of success? 
How often do you highlight these stories to others? 


It is critical that we 
consistently examine how 

we ate ensuring that our 
assessment truths that we 
create and the practices we 
employ are in alignment as we 
look forward to advancing the 
next chapter of assessment. 
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¢ Take a deeper look. How do you position faculty in relationship to assessment? 
How have your messages drawn upon shared values and beliefs about learning? 
How do you know what those shared values are? 

¢ Who may have been unintentionally excluded from communications and 
conversations? To what degree are partners whose perspectives differ from yours 
included? 


From Foundational Truths to Practice 

Another way to look for alignment between foundational truths of assessment and 
solutions is to examine your assessment office’s vision statement, communication to 
faculty and administrators, and assessment practices and policies (report templates, data 
discussions, assessment infrastructure). If one of your foundational truths is centered on 
a commitment to advancing equitable learning opportunities for students, think about 
how you will respond when the next faculty or administrator asks, “What (fill in your 
accreditor of choice) needs from us to be in compliance?” When sharing assessment 
results, how will you lead the conversation toward equitable student learning, as opposed 
to strictly reporting of results? Are students included in the communication chain? How 
will you make assessment data useful and meaningful for your audiences? Whose voice 
should be narrating or co-narrating these stories about student learning? 


It is critical that we consistently examine that our assessment truths and practices we 
employ are in alignment as we look forward to advancing the next chapter of assessment. 
Just as we work to ensure curricular alignment between learning outcomes and learning 
opportunities, we can map how our foundation truths of assessment align with our 
solutions and practices. Here’s one approach to examining for alignment (Table 1). 


Foundational Truths New Solutions and Practices 
Assessment is about students and student Ensure learning outcomes are transparent 
learning, (See NILOA Transparency Framework). 
Assessment creates space for critical Ensure assessment practices leave room 
reflection and action. for discussion, reflection, and action. 


Be transparent about what actions are 
taking place as a result of assessment 


findings. 
Assessment opens up an opportunity to — Build assessment practices and policies 
engage in equity-minded sense-making that are inclusive and bias-free. 
(McNair et al., 2020). 


Interrogate and disaggregate data and 
include multiple data points ot ensure 
a complete picture of student learning 
(Montenegro & Jankowski, 2020). 


Student voice is integral to the work of Actively involve students as partners in 


assessment. assessment that include constructing 


assessment tools to data sense-making. 


Table 1. Moving From Truths to Practice. 
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In this section, we offered initial approaches to interrogating and recentering our 
assessment practices through the first principles development approach that identifies 
assumptions, grounds our work on non-negotiable truths, and forms new solutions 
and practices that are aligned. We posited that engaging in this reflextve practice is a 
necessary first step toward deconstructing legacy assumptions, received practices, and 
counterproductive narratives that have derailed why we engage in the work. Lastly, we 
challenge assessment leaders to critically exam their own campus assessment cultures and 
courageously deconstruct previously unquestioned practices that have gotten in the way 
of attending to learning inequities at their institutions. 


Moving Forward Together 


If the disruption to institutional norms and the emerging consciousness of racism in our 
nation has illuminated anything in higher education assessment, it has shone a light on 
a need for a new way forward that is rooted in critical reflection, followed by rebuilding 
practices attuned to the realities of our students today. True transformation cannot be 
built upon old foundations. New foundations are needed. 


As we establish new assessment practices and cultures as a result of engaging in this 
reflection-deconstruction-action process, we must be mindful that change initiatives 
may privilege some interests over others. If our change initiatives are grounded in 
students’ interests and ultimately intended to benefit students, allow that conviction to 
guide assessment practices (Kezar, 2013). Critical to the reconstruction of assessment 
practices that moves beyond the compliance versus continuous improvement binary is 
the intentional alignment between principal truths, discourses, and practices centered 
around students. We reject dichotomized views of assessment as either accountability or 
improvement when talking about the purpose of assessment work. If we allow ourselves 
to be seduced into arguing false equivalencies, we run the risk of being distracted from 
the very reason we do our work: students. 


Lastly, we are attuned to the fact that not every assessment professional holds positional 
power or authority in their institutions to make transformational changes. We contest the 
idea that individuals cannot make a difference. Similarly, as assessment professionals we 
likely will not be the ones who will be creating, testing, approving, and administering a 
vaccine for COVID-19 to confront and halt this pandemic. However, we have the power 
to put on a mask and prevent the spread of this virus. While we alone will not be able to 
end racism in our institutions, we hold the power to examine whether our own words 
and actions are perpetuating racism and commit to be anti-racist ourselves. We assert 
that we all hold some level of power and privilege in shaping values, practices, policies, 
and moreover, narratives around student learning assessment. It is up to us to lead the 
rebuilding of assessment that supports all learners whose voices and stories we amplify. 
A new assessment movement is on the horizon that pushes beyond the compliance 
versus continuous improvement binary and toward a third way forward rooted in critical 
consciousness and responsive to educational inequity. 


If the disruption to 
institutional norms and the 
emerging consciousness of 
tacism in our nation has 
illuminated anything in higher 
education assessment, it has 
shone a light on a need for 

a new way forward that is 
rooted in critical reflection, 
followed by rebuilding 
practices attuned to the 
realities of our students today. 
True transformation cannot be 
built upon old foundations. 
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